Improving the use of research evidence in guideline development: 15. Disseminating and implementing guidelines

BACKGROUND: The World Health Organization (WHO), like many other organisations around the world, has recognised the need to use more rigorous processes to ensure that health care recommendations are informed by the best available research evidence. This is the 15(th )of a series of 16 reviews that have been prepared as background for advice from the WHO Advisory Committee on Health Research to WHO on how to achieve this. OBJECTIVES: In this review we address strategies for the implementation of recommendations in health care. METHODS: We examined overviews of systematic reviews of interventions to improve health care delivery and health care systems prepared by the Cochrane Effective Practice and Organisation of Care (EPOC) group. We also conducted searches using PubMed and three databases of methodological studies for existing systematic reviews and relevant methodological research. We did not conduct systematic reviews ourselves. Our conclusions are based on the available evidence, consideration of what WHO and other organisations are doing and logical arguments. KEY QUESTIONS AND ANSWERS: What should WHO do to disseminate and facilitate the uptake of recommendations? • WHO should choose strategies to implement their guidelines from among those which have been evaluated positively in the published literature on implementation research • Because the evidence base is weak and modest to moderate effects, at best, can be anticipated, WHO should promote rigorous evaluations of implementation strategies. What should be done at headquarters, by regional offices and in countries? • Adaptation and implementation of WHO guidelines should be done locally, at the national or sub-national level. • WHO headquarters and regional offices should support the development and evaluation of implementation strategies by local authorities

Improving the use of research evidence in guideline development: 16. Evaluation

BACKGROUND: The World Health Organization (WHO), like many other organisations around the world, has recognised the need to use more rigorous processes to ensure that health care recommendations are informed by the best available research evidence. This is the last of a series of 16 reviews that have been prepared as background for advice from the WHO Advisory Committee on Health Research to WHO on how to achieve this. OBJECTIVES: We reviewed the literature on evaluating guidelines and recommendations, including their quality, whether they are likely to be up-to-date, and their implementation. We also considered the role of guideline developers in undertaking evaluations that are needed to inform recommendations. METHODS: We searched PubMed and three databases of methodological studies for existing systematic reviews and relevant methodological research. We did not conduct systematic reviews ourselves. Our conclusions are based on the available evidence, consideration of what WHO and other organisations are doing and logical arguments. KEY QUESTIONS AND ANSWERS: Our answers to these questions were informed by a review of instruments for evaluating guidelines, several studies of the need for updating guidelines, discussions of the pros and cons of different research designs for evaluating the implementation of guidelines, and consideration of the use of uncertainties identified in systematic reviews to set research priorities. How should the quality of guidelines or recommendations be appraised? • WHO should put into place processes to ensure that both internal and external review of guidelines is undertaken routinely. • A checklist, such as the AGREE instrument, should be used. • The checklist should be adapted and tested to ensure that it is suitable to the broad range of recommendations that WHO produces, including public health and health policy recommendations, and that it includes questions about equity and other items that are particularly important for WHO guidelines. When should guidelines or recommendations be updated? • Processes should be put into place to ensure that guidelines are monitored routinely to determine if they are in need of updating. • People who are familiar with the topic, such as Cochrane review groups, should do focused, routine searches for new research that would require revision of the guideline. • Periodic review of guidelines by experts not involved in developing the guidelines should also be considered. • Consideration should be given to establishing guideline panels that are ongoing, to facilitate routine updating, with members serving fixed periods with a rotating membership. How should the impact of guidelines or recommendations be evaluated? • WHO headquarters and regional offices should support member states and those responsible for policy decisions and implementation to evaluate the impact of their decisions and actions by providing advice regarding impact assessment, practical support and coordination of efforts. • Before-after evaluations should be used cautiously and when there are important uncertainties regarding the effects of a policy or its implementation, randomised evaluations should be used when possible. What responsibility should WHO take for ensuring that important uncertainties are addressed by future research when the evidence needed to inform recommendations is lacking? • Guideline panels should routinely identify important uncertainties and research priorities. This source of potential priorities for research should be used systematically to inform priority-setting processes for global research

Developing the content of two behavioural interventions : using theory-based interventions to promote GP management of upper respiratory tract infection without prescribing antibiotics #1

Background: Evidence shows that antibiotics have limited effectiveness in the management of upper respiratory tract infection (URTI) yet GPs continue to prescribe antibiotics. Implementation research does not currently provide a strong evidence base to guide the choice of interventions to promote the uptake of such evidence-based practice by health professionals. While systematic reviews demonstrate that interventions to change clinical practice can be effective, heterogeneity between studies hinders generalisation to routine practice. Psychological models of behaviour change that have been used successfully to predict variation in behaviour in the general population can also predict the clinical behaviour of healthcare professionals. The purpose of this study was to design two theoretically-based interventions to promote the management of upper respiratory tract infection (URTI) without prescribing antibiotics. Method: Interventions were developed using a systematic, empirically informed approach in which we: selected theoretical frameworks; identified modifiable behavioural antecedents that predicted GPs intended and actual management of URTI; mapped these target antecedents on to evidence-based behaviour change techniques; and operationalised intervention components in a format suitable for delivery by postal questionnaire. Results: We identified two psychological constructs that predicted GP management of URTI: "Self-efficacy," representing belief in one's capabilities, and "Anticipated consequences," representing beliefs about the consequences of one's actions. Behavioural techniques known to be effective in changing these beliefs were used in the design of two paper-based, interactive interventions. Intervention 1 targeted self-efficacy and required GPs to consider progressively more difficult situations in a "graded task" and to develop an "action plan" of what to do when next presented with one of these situations. Intervention 2 targeted anticipated consequences and required GPs to respond to a "persuasive communication" containing a series of pictures representing the consequences of managing URTI with and without antibiotics. Conclusion: It is feasible to systematically develop theoretically-based interventions to change professional practice. Two interventions were designed that differentially target generalisable constructs predictive of GP management of URTI. Our detailed and scientific rationale for the choice and design of our interventions will provide a basis for understanding any effects identified in their evaluation. Trial registration: Clinicaltrials.gov NCT00376142This study is funded by the European Commission Research Directorate as part of a multi-partner program: Research Based Education and Quality Improvement (ReBEQI): A Framework and tools to develop effective quality improvement programs in European healthcare. (Proposal No: QLRT-2001-00657)

Translating clinicians' beliefs into implementation interventions (TRACII) : a protocol for an intervention modeling experiment to change clinicians' intentions to implement evidence-based practice

Background: Biomedical research constantly produces new findings, but these are not routinely incorporated into health care practice. Currently, a range of interventions to promote the uptake of emerging evidence are available. While their effectiveness has been tested in pragmatic trials, these do not form a basis from which to generalise to routine care settings. Implementation research is the scientific study of methods to promote the uptake of research findings, and hence to reduce inappropriate care. As clinical practice is a form of human behaviour, theories of human behaviour that have proved to be useful in other settings offer a basis for developing a scientific rationale for the choice of interventions. Aims: The aims of this protocol are 1) to develop interventions to change beliefs that have already been identified as antecedents to antibiotic prescribing for sore throats, and 2) to experimentally evaluate these interventions to identify those that have the largest impact on behavioural intention and behavioural simulation. Design: The clinical focus for this work will be the management of uncomplicated sore throat in general practice. Symptoms of upper respiratory tract infections are common presenting features in primary care. They are frequently treated with antibiotics, and research evidence is clear that antibiotic treatment offers little or no benefit to otherwise healthy adult patients. Reducing antibiotic prescribing in the community by the "prudent" use of antibiotics is seen as one way to slow the rise in antibiotic resistance, and appears safe, at least in children. However, our understanding of how to do this is limited. Participants will be general medical practitioners. Two theory-based interventions will be designed to address the discriminant beliefs in the prescribing of antibiotics for sore throat, using empirically derived resources. The interventions will be evaluated in a 2 × 2 factorial randomised controlled trial delivered in a postal questionnaire survey. Two outcome measures will be assessed: behavioural intention and behavioural simulation.This study is funded by the European Commission Research Directorate as part of a multi-partner program: Research Based Education and Quality Improvement (ReBEQI): A Framework and tools to develop effective quality improvement programs in European healthcare. (Proposal No: QLRT-2001-00657)

SUPPORT Tools for evidence-informed health Policymaking (STP) 5: Using research evidence to frame options to address a problem

This article is part of a series written for people responsible for making decisions about health policies and programmes and for those who support these decision makers

Does updating improve the methodological and reporting quality of systematic reviews?

BACKGROUND: Systematic reviews (SRs) must be of high quality. The purpose of our research was to compare the methodological and reporting quality of original versus updated Cochrane SRs to determine whether updating had improved these two quality dimensions. METHODS: We identifed updated Cochrane SRs published in issue 4, 2002 of the Cochrane Library. We assessed the updated and original versions of the SRs using two instruments: the 10 item enhanced Overview Quality Assessment Questionnaire (OQAQ), and an 18-item reporting quality checklist and flow chart based upon the Quality of Reporting of Meta-analyses (QUOROM) statement. At least two reviewers extracted data and assessed quality. We calculated the percentage (with a 95% confidence interval) of 'yes' answers to each question. We calculated mean differences in percentage, 95% confidence intervals and p-values for each of the individual items and the overall methodological quality score of the updated and pre-updated versions using OQAQ. RESULTS: We assessed 53 SRs. There was no significant improvement in the global quality score of the OQAQ (mean difference 0.11 (-0.28; 0.70 p = 0.52)). Updated reviews showed a significant improvement of 18.9 (7.2; 30.6 p < .01) on the OQAQ item assessing whether the conclusions drawn by the author(s) were supported by the data and/or analysis presented in the SR. The QUOROM statement showed that the quality of reporting of Cochrane reviews improved in some areas with updating. Improvements were seen on the items relating to data sources reported in the abstract, with a significant difference of 17.0 (9.8; 28.7 p = 0.01), review methods, reported in the abstract 35 (24.1; 49.1 p = 0.00), searching methods 18.9 (9.7; 31.6 p = 0.01), and data abstraction 18.9 (11.7; 30.9 p = 0.00). CONCLUSION: The overall quality of Cochrane SRs is fair-to-good. Although reporting quality improved on certain individual items there was no overall improvement seen with updating and methodological quality remained unchanged. Further improvement of quality of reporting is possible. There is room for improvement of methodological quality as well. Authors updating reviews should address identified methodological or reporting weaknesses. We recommend to give full attention to both quality domains when updating SRs

Implementation of an evidence-based guideline on fluid resuscitation: lessons learnt for future guidelines

There is little experience with the nationwide implementation of an evidence-based pediatric guideline on first-choice fluid for resuscitation in hypovolemia. We investigated fluid prescribing behavior at (1) guideline development, (2) after guideline development, and (3) after active implementation and identified potential barriers and facilitators for guideline implementation. In order to minimize costs and to optimize implementation effect, we continuously developed and adjusted implementation strategies according to identified barriers. Implementation success was evaluated using questionnaires, pharmaceutical data, and data from medical records. The most remarkable change occurred after guideline development and dissemination: Normal saline use by neonatologists increased from 22-89% to 100% and by pediatric intensivists from 43-71% to 88-100%, and synthetic colloid use by pediatric intensivists declined from 29-43% to 0-13% with a reduction in albumin use by neonatologists from 11-44% to 0%. After active guideline implementation, most of specialist's management behavior was according to the guideline. Stakeholders involved in the developmental process are of great importance in disseminating recommendations before active implementation. Therefore, to successfully implement guidelines and reduce costs of active implementation, any guideline development should consider implementation right from the beginning. Implementation strategies should target identified barriers and will therefore always be guideline specifi

A pragmatic cluster randomised trial evaluating three implementation interventions

Background Implementation research is concerned with bridging the gap between evidence and practice through the study of methods to promote the uptake of research into routine practice. Good quality evidence has been summarised into guideline recommendations to show that peri-operative fasting times could be considerably shorter than patients currently experience. The objective of this trial was to evaluate the effectiveness of three strategies for the implementation of recommendations about peri-operative fasting. Methods A pragmatic cluster randomised trial underpinned by the PARIHS framework was conducted during 2006 to 2009 with a national sample of UK hospitals using time series with mixed methods process evaluation and cost analysis. Hospitals were randomised to one of three interventions: standard dissemination (SD) of a guideline package, SD plus a web-based resource championed by an opinion leader, and SD plus plan-do-study-act (PDSA). The primary outcome was duration of fluid fast prior to induction of anaesthesia. Secondary outcomes included duration of food fast, patients' experiences, and stakeholders' experiences of implementation, including influences. ANOVA was used to test differences over time and interventions. Results Nineteen acute NHS hospitals participated. Across timepoints, 3,505 duration of fasting observations were recorded. No significant effect of the interventions was observed for either fluid or food fasting times. The effect size was 0.33 for the web-based intervention compared to SD alone for the change in fluid fasting and was 0.12 for PDSA compared to SD alone. The process evaluation showed different types of impact, including changes to practices, policies, and attitudes. A rich picture of the implementation challenges emerged, including inter-professional tensions and a lack of clarity for decision-making authority and responsibility. Conclusions This was a large, complex study and one of the first national randomised controlled trials conducted within acute care in implementation research. The evidence base for fasting practice was accepted by those participating in this study and the messages from it simple; however, implementation and practical challenges influenced the interventions' impact. A set of conditions for implementation emerges from the findings of this study, which are presented as theoretically transferable propositions that have international relevance. Trial registration ISRCTN18046709 - Peri-operative Implementation Study Evaluation (POISE

Timescales of Massive Human Entrainment

The past two decades have seen an upsurge of interest in the collective behaviors of complex systems composed of many agents entrained to each other and to external events. In this paper, we extend concepts of entrainment to the dynamics of human collective attention. We conducted a detailed investigation of the unfolding of human entrainment - as expressed by the content and patterns of hundreds of thousands of messages on Twitter - during the 2012 US presidential debates. By time locking these data sources, we quantify the impact of the unfolding debate on human attention. We show that collective social behavior covaries second-by-second to the interactional dynamics of the debates: A candidate speaking induces rapid increases in mentions of his name on social media and decreases in mentions of the other candidate. Moreover, interruptions by an interlocutor increase the attention received. We also highlight a distinct time scale for the impact of salient moments in the debate: Mentions in social media start within 5-10 seconds after the moment; peak at approximately one minute; and slowly decay in a consistent fashion across well-known events during the debates. Finally, we show that public attention after an initial burst slowly decays through the course of the debates. Thus we demonstrate that large-scale human entrainment may hold across a number of distinct scales, in an exquisitely time-locked fashion. The methods and results pave the way for careful study of the dynamics and mechanisms of large-scale human entrainment.Comment: 20 pages, 7 figures, 6 tables, 4 supplementary figures. 2nd version revised according to peer reviewers' comments: more detailed explanation of the methods, and grounding of the hypothese

SUPPORT Tools for evidence-informed health Policymaking (STP) 8: Deciding how much confidence to place in a systematic review

This article is part of a series written for people responsible for making decisions about health policies and programmes and for those who support these decision makers